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CN ' Abstract 

We present a multivariate central limit theorem for a general class of interacting 
Markov chain Monte Carlo algorithms used to solve nonlinear measure- valued equations. 
These algorithms generate stochastic processes which belong to the class of nonlinear 
Markov chains interacting with their empirical occupation measures. We develop an 
original theoretical analysis based on resolvent operators and semigroup techniques to 
Oh analyze the fluctuations of their occupation measures around their limiting values. 

Oh ' Keywords : Multivariate central limit theorems, random fields, martingale limit 

theorems, self-interacting Markov chains, Markov chain Monte Carlo algorithms. 

Mathematics Subject Classification : 
Primary: 47H20, 60G35, 60J85, 62G09; Secondary: 47D08, 47G10, 62L20. 



1 Introduction 

> , 

1.1 Nonlinear measure- valued equations 

Let (£0 ,5^)(>o be a sequence of measurable spaces. For any I > we denote by V(S^) 
the set of probability measures on ■ Suppose we have a sequence of probability measures 
tj-W £ V(S^) such that, for any I > 1, tt^ satisfies the following nonlinear measure- valued 
CN j equation 

$0(^-1)) = 7r (0 (1.1) 

for some mappings 

$(0 . p(S( l -V) -»■ V(S®). We will also use the convention &(°\tt( 1 )) = 

7r(°). 

In numerous scenarios, the probability measures (tt^)i>o need to be approximated nu- 
merically. Interacting particle methods have been previously proposed to approximate these 
probability distributions [HHO]- However they suffer from several limitations detailed in [7], 
[8]. To bypass some of these limitations, an alternative class of algorithms known as inter- 
acting Markov chain Monte Carlo (i-MCMC) methods has been recently introduced in |7|, 
[8]. The main objective of this article is to present a multivariate Central Limit Theorem 
(CLT) for these methods. This extends significantly our previous result established in [6] 
which only applies to a restricted class of i-MCMC algorithms. 

Before describing two general class of models where i-MCMC methods can be used, we 
introduce the notation adopted in this paper. 
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1.2 Notation and conventions 

We denote respectively by M(E), Mq(E), V(E), and B(E), the set of all finite signed 
measures on some measurable space (E,£) equipped with some a- field £, the convex subset 
of measures with null mass, the subset of all probability measures, and finally the Banach 
space of all bounded and measurable functions / on E equipped with the uniform norm 
ll/H = sup xGE |/(x)| and the Borel a-field associated to the supremum norm. We also 
denote by By(E) C B(E) the unit ball of functions / G B(E) with ||/|| < 1, and by Osci(£), 
the convex set of ^-measurable functions / with oscillations less than one; that is, 

osc(/) = sup{|/(x)-/(y)| ;x,yeE}<l 

We let n(f) = f n(dx) f(x) be the Lebesgue integral of a function / G B(E) with 
respect to a measure [i G M{E). We slightly abuse the notation, and sometimes we denote 
by /J.(A) = h(1a) the measure of a measurable subset A G £. 

We recall that a bounded integral operator M from a measurable space (E, £) into an 
auxiliary measurable space (F, F) is an operator / >->■ M(/) from B{F) into B{E) so that 
the functions 

M(f){x) = J M(x,dy) f(y) GM 

are ^-measurable and bounded for any / G B(F). By Fubini's theorem, we recall that a 
bounded integral operator M from a measurable space (E, £) into an auxiliary measurable 
space (F,F) also generates a dual operator fi H> fiM from M{E) into M(F) defined by 
( M M)(/):= M (M(/)). 

We denote by ||M|| := supj eBl( -£) ||M(/)|| the norm of the operator / >->■ M(f) and 
we equip the Banach space M{E) with the corresponding total variation norm ||/x|| = 
su VfeB 1 (E) Im(/)I- We let (3(M) be the Dobrushin coefficient of a bounded integral operator 
M defined by the following formula 

P(M) := sup{osc(M(/)) ; / G Os Cl (F)} 

When M has a constant mass, that is M(l)(x) = M(l)(y) for any (x,y) G E 2 , the operator 
[i I—?- /iM maps Mq{E) into A^o(-^) an d (3(M) coincides with the norm of this operator. 
We also denote by (M fc )fc>o the semigroup associated to M given by the recursive formulae 
M k (x,dz) = J M k ~ 1 (x,dy)M(y,dz), for k > 1 and M° = Id the identity transition. 

For any sequence of finite signed measures {Hi) i>0 defined on some collection of measur- 
able spaces (Ei,£i) i>0 , we define (/Xj fij) (dxi,dxj) = Hi{dxi) fij (dxj), /Uj® 2 (dxi, dx'A = 
(m <g> Hi) (dxi, dx'A and 

3 

®i<k<jHk (dxi, dxi+i, . . . ,dxj) = Y[^k {dx k ) . 

k=i 

For two bounded measurable functions / and g defined respectively on (E,£) and (F,F) 
we define (/ (g) g) (x, x') = f (x) g (x'). 

We equip the set of distribution flows M(E) N with the uniform total variation distance 
defined by 

V?? = (%)n>0, A* = {Vn)n>o G M{E) N \\rj - fi\\ := sup \\r) n - Hn\\ 

n>0 

We extend a given bounded integral operator /x G M(E) >->■ [iM G M(F) into an mapping 
r/ = (Vn)n>o G A^(i?) N M- 7/£> = (r/ n M) n > G A^(F) N 
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Sometimes, we abuse the notation and denote by v instead of (v) n >o the constant distribu- 
tion flow equal to a given measure v E V{E). 

For any M^-valued function / = (f l )\<i<d G B(F) d , any integral operator M from E 
into F, and any \jl G A4(F), we will slightly abuse the notation, and we write M(f) and 
/J,(f) the Revalued function and the point in M. d given respectively by 

M(f) := (Ml/ 1 ), . . . , M{f d )) and := (/.(J 1 ), . . . , //(/<*)) 

We also simplify the notation and sometimes we write 

M^-MU 1 )) (f 2 -M(f))](x) 

instead of 

M[(f - M(/!)(x)) (/ 2 - M(/ 2 )(x))](x) = M(/V 2 )(x) - M(/!)(x) M(/ 2 )(x) 

Unless otherwise is stated, we denote by c(k), k G N, a constant whose value may vary from 
line to line but only depends on the parameter k. Finally, we shall use = and J7j = 1. 



1.3 Examples 

We give here two classes of models where i-MCMC methods can be used. 
Feynman-Kac models. In this context, we have 

VZ > V(m, /) € (V(S®) x B(S^)) * (m ■= /x(G,L,+i(/))/m(G,) (1.2) 

where Li + \ is a Markov transition kernel from into and Gi : S« -»• M+. In this 

situation, the solution of the measure-valued equation is given by 

vr (0 (/)=7 (0 (/)/7 (0 (l) with 7 (0(/): = E[/pQ) [J G*(^) (1.3) 

\ o<fc<« / 

where (^;);>o is a Markov chain taking values in the state spaces (S®)i>o, with initial 
distribution 7P°) and Markov transitions (L/)/>x- These Feynman-Kac models arise in a 
large number of applications including nonlinear filtering, Bayesian statistics and physics; 
see [9l [TO]. Note that these models are quite flexible. For instance, the reference Markov 
chain may represent the paths from the origin up to the current time I of an auxiliary 
Markov chain (-^/); >0 taking values in some state spaces (<S[)j >0 with initial distribution 
7r(°) and Markov transitions (L',)i>i; that is, we have 



X l := (X' , . . . , X[) G S® := (S' X . . . X 5f) (1.4) 



and consequently 



U (xi-^dyi) = (dyi-i)L[ (y'l^dyl) (1.5) 

When G[(xi) = G'Jx',), that is the potential function only depends on the terminal value of 
the path xi = (x' , . . . , xj), the measures correspond to the path space measures given 
for I > 1 by 

7T«(^)oC \ H G' k (x' k ) \ 7T^(dx' ) ]^[ 4(4_!,d4) (1.6) 
0<k<l Kk<l 
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where 'oc' means 'proportional to'. ■ 

Interacting annealing models. These models were recently introduced in [3] and can 
be reinterpreted as a special case of (jl.ip . In this scenario, we have = S and a pre- 
determined sequence of probability distributions (tt^)i>q of the form 

w _ exp (-PjV (a) ) Xjdx) 
[ } ~ A(exp(-/W) 
where A is a reference measure, (A);> is an increasing positive sequence and A (exp (— ^ V)) < 
oo. Based on this sequence, we build a sequence of mappings [®"')i >0 satisfying (jl.ip as 
follows. We introduce e G [0,1) and two sequences of Markov kernels (Ki) l>0 and (Li) l>0 
where both K\ and L\ admit tt^ 1 ' as invariant measure. We then set 

VI > V(m,/) G x B(S^)) & l+1 Hp) (/) := ^(/^z+i^+i (/) (1-7) 

where f, : /i£ h-> $ z (/z) g 7>(S) is defined by 

- (1-8) 

for (x) = exp (-(A+l - A)V (x)) and 



K 6>l :=(l-e) £V 



k>0 

It is easy to check that (jl.ip is satisfied for the mappings (jl.7p , ■ 
1.4 Interacting Markov chain Monte Carlo methods 

We introduce a sequence of 'initial' probability measures ) z>0 on );>o" ^ e a ^ so 
introduce a Markov transition from into itself, and a collection of Markov transi- 
tions Rljl from into itself, indexed by the parameter I > 0, where fi G V{S^ 1 ^). We 
further assume that the invariant measure of each operator is given by $>W(/i); that is 
we have 

VI >0 V/iGP(S( w )) $ W (//) = $«(//) AfW 

For / = 0, we use the convention = and = tt (0) . 

We have now all the elements to define the i-MCMC algorithm. The algorithm generates 
a sequence of processes (X®) l>Q where := (Xn)n>0 is the process at level k whose 
associated occupation measure at iteration n of the algorithm is denoted by 



„(*) - = 

In n + _ 

p=0 



1 n 



At level fc = 0, is a Markov chain on with Xq ~ and Markov transitions 
M(°);thatis 

G dx | 4°)) = M(°) (X(°),dx) . 

At level > 1, given a realization of the chain X^ -1 ' , the fc-th level chain is an 

inhomogeneous Markov chain with -Xq ~ z/ fc ) and Markov transitions M^_ 1} at iteration 

Vn 

n depending on the current occupation measure rjn ^ of the chain at level (k — 1); that is 
P(Afi G dx | X^\ X«) = M^^W,^). (1.9) 

?7n 
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(k) 

The rationale behind this is that the k-ih level chain X„ behaves asymptotically as an 

(k) 

homogeneous Markov chain with transition kernel M K ^_^ of invariant probability measure 

7r^ as long as rjn ^ is a 'good' approximation of tt^ 1 ^. 

These i-MCMC algorithms can be interpreted as non-standard adaptive MCMC schemes 
[H El H5] where the parameters to be adapted are probability measures instead of finite- 
dimensional parameters. Algorithms relying on similar principles were first proposed in 
[3] and independently in [4]. Related algorithms where we also have a sequence of nested 
MCMC-like chains 'feeding' each other have also recently appeared in statistics |12| and 
physics [13] . 

We now give examples of such Markov kernels for the Feynman-Kac and interacting 
annealing models described in section fT73l 

Feynman-Kac models. Assume we are working on path spaces := (S^ 1 ' x S',) (cf. 

(jl.4j) - (jl.6p ). we can select for a Metropolis-Hastings kernel of independent proposal 
distribution (fx® L[) and target distribution <E>W(/z). More precisely, using the fact that 

the independent Metropolis-Hastings kernel using /j,(dyi-i) L' l (y' l _ l , dy[) as a proposal 
distribution is given by 

M®(x u d Vl ) 

(1.10) 

= tidm-i)miLvM) ( 1A feSB) + ( 1 -K 1A fefer)) 5 ^ dyi) 

where we recall that xi = (xz_2, arj) = {xi-i,x[) € = (S( l ~^ x S' t ) and y\ = 

(m^yl) e = (SV-V x s>). u 

Interacting annealing models. In this case, we can select 

Mjp(x,dy) = eK l (x,dy) + {l-e) Vi-i{p)Li(dy). (1.11) 

One can easily check that M$ admits 3>(')(/i) as invariant probability measure. ■ 

For sufficiently regular models, we proved in [3 E] that the occupations measures rjn 
converge to the solution of the equation (jl.ip . in the sense that lim n ,_ > . 00 rjn (f) = vr^(/) 
almost surely for / G B(S^). The articles [HE] also provide a collection of non asymptotic 
L r -mean error estimates and exponential deviations inequalities. The fluctuation analysis 
of r]n around the limiting measure has been initiated in [6] in the special case where 
M/p(xi, .) = & l \fj,). In this 'simpler' situation, the l-th level chain X® = fx®} is given 

X^ 1 } a collection of conditionally independent random variables with X^ ~ i/W and 
X n l) ~ <5> {l) {Vn-i) for n > 1. 

1.5 Contribution and organization of the paper 

The present article studies the fluctuations of the occupation measures (ijn^) associated 



l>0 

to the class of i-MCMC algorithms towards their limiting values ( 7T ^)i >0 - Briefly speaking, 
our analysis proceeds as follows. First, we study weighted sequences of local random fields 
Vn which are related to the fluctuations of the occupation measures 77^ around their 



local invariant measures $0 (i]p_P J for p < n. We show that these random fields (Vn 



converges in law, as n tends to infinity and in the sense of finite dimensional distributions, to 
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a sequence of independent and centered Gaussian fields {V^) l>0 with covariance functions 

denned in terms of the resolvent operator associated to the Markov transition M and 

its invariant probability measure tt^. Finally, we deduce the fluctuations of rjip around 
their limiting values tt^ by a simple application of the continuous mapping theorem (or the 

multivariate 5-method) applied to a first order decomposition of the error \fn 

(k) 

in terms of the random fields (Vn )o<k<i- 

The rest of the paper is organized as follows. The main result of the article is presented 
in full details in section [2j The regularity conditions are summarized in section 12.11 In 
section 12.21 we state a multivariate CLT in terms of the semigroup associated with a first 
order expansion of the mappings $0 appearing in (jl.ip . Section [3] addresses the fluctuation 
analysis of an abstract class of time inhomogeneous Markov chains. In section 13. 2\ we 
present a preliminary resolvent analysis to estimate the regularity properties of resolvent 
operators and invariant measure type mappings. In section 13.31 we apply these results to 
study the local fluctuations of a class of weighted occupation measures associated to self- 
interacting chains. Section 0] addresses the fluctuation analysis of local interaction random 
fields associated with i-MCMC algorithms. The proof of the main theorem presented in 
section 12.21 is a direct consequence of a fluctuation theorem for local interaction random 
fields, and it is given at the end of section 14.11 Finally, we establish in section [5] that the 
regularity conditions discussed in section 12.11 are also valid for a path space extension of 
i-MCMC algorithms. 



2 Statement of some results 
2.1 Regularity conditions 

Our first regularity condition is a first order weak regularity condition on the mappings 
governing the measure- valued equation (jl.ip . We assume that, for any I > 0, the mappings 
<j)('+ 1 ) : 'P^S^) — > 'P(S'^ +1 )) satisfy the following first order local decomposition 

<& (l+1 V)-^ (i+1) (»7)] =(ji-v)Di + i + E l (p, V ) (2.1) 

where D l+1 : B(S r P+ 1 )) -> B(S®) is a bounded integral operator that may depend on 
the measure r\ and H/(/i, rf) is a remainder signed measure on S^ l+1 ^ indexed by the set of 
probability measures fJ>,rj G V(S^). We further require that 

|Sz(/^)(/)| < J \(»- V f 2 (9)\ Sfodg) (2.2) 

for some integral operator Si from B(S^ l+1 ^) into the set T^iS^) of all tensor product 
functions g = £ ie/ <n (hj ® hj), with / C N, (hj,hj) ieI G (B(S®) 2 ) Z , and a sequence of 
numbers (aj)j g / £ WL 1 such that 

\d\ -=y2 \ai\ \\hj\\\\hf\\ < oo and xi ■= sup / \g\ Ei(f, dg) < oo (2.3) 

Our second set of regularity conditions are for the Markov kernels Affp. We assume 
these kernels satisfy the following two regularity conditions 

771^):= sup /3((M«)"0<1 (2.4) 
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and 



M W - Af W 



(/) < / r^(/,d 5 ) (2.5) 



for some collection of bounded integral operators Ti tfl from B(S^) into ^) and indexed 

by the set of measures ji G P(5'^~ 1 -') with 

sup / Ti ifi (f,dg) \\g\\ < A; ||/|| and A; < oo 
^(Sti-i)) J 

We end this section with some comments on this set of conditions. 

The regularity condition (|2.ip -( l2~?2~j) is a first order refinement of a Lipschitz type condi- 
tion we used in [8] to derive a series of L p -mean error bounds and exponential inequalities. 
This condition has been introduced in [6 J for studying the fluctuations of the simple i-MCMC 
algorithm corresponding to M^\x, .) = <&^(/i). The regularity condition (I2.4p is an ergod- 
icity condition on the Markov kernel Afjp . Finally the regularity condition (j2.5[) is a local 
Lipschitz type continuity condition on the kernel Mjp . This condition is less stringent than 
the one used in [8] where it is assumed that (|2.5p holds for some operators „ = T[ that do 
not depend on fj,. Therefore, most of the asymptotic results presented in [8] do not apply in 
the present context. Nevertheless, it can be checked that the inductive proof of the L p -mean 
error bounds presented in theorem 5.2 in [6] hold true under the weaker condition (|2.5|) : thus, 
for every I > and any function / G B(S^), we know that r)n\f) converges almost surely 
to 7P^(/) as n — > oo. The main advantage of the set of conditions presented here is that it 
is stable under a state space enlargement (see Section [5]) , so that the asymptotic analysis 
of such algorithms, including the multivariate CLT presented in the next section, applies 
directly without further work to i-MCMC algorithms on path spaces. 

We illustrate these regularity conditions for the models discussed in section 11.31 We 
further assume in the rest of this section that (Gi) l>0 is a collection of (0, l]-valued potential 
functions on some state space (S^) l>Q such that 

V/ > inf Gi > (2.6) 

Feynman-Kac models. To establish (|2.ip - (|2.2p . we observe that the mapping ^ defined 
in (|1.8p can be rewritten in terms of a nonlinear transport equation 



^i{^){dy) = (pSl,n) (dy) := J ^(dx)Si ifjL (x } dy) 

where 

S ltll (x,dy) = Gi(x) 5 x (dy) + (1 - G,(a;)) %(p)(dy) 
Using the decomposition 

- tyfa) = (/i - ri)S ltV + fi(S ltli - S l)V ) - tyfa) = (/i - r,)S ljV (2.7) 

V(Gi) 

we prove the first order decomposition 

- ®i(v) = (M - V)DI + 3/_i(M,?7) 

with the integral operators D[ defined for any / G B(S"') by D[(f) := (r/(G;)) _1 Si jTj (f), 
and the remainder measures 



SUMO) == 



1 



L/^) »/(G,) 



(a* - v)Si,M 
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Using the fact that 



KGi) v(Gi) 



and 



we obtain 



Ou - v )SiM = (j*-v) (Gif) - (/x - rim) Mv)(f) 



t-iM(f)\ < ^ ~ l?) " 2(G, ® (G/))| + 11/11 l°" ~ v)m{Gl ® G,) D 



($G l ®(G l f)(dg) + 11/11 Sa^Giidg)) 



with the integral operator 

Sj-i(/,<fc) 

We check that the mappings (|1.2|) satisfy (|2.ip using the fact that 

_ $ G+i) (r?) = _ y l{v))Ll+1 = fa — r,)D l+1 + S,(M,r/) (2.8) 

with the first order operator -D/+1 = D[Li + i and the remainder measure 77) = Sj_ 1 (/i, r/)L/ + i. 
The remainder measure satisfies (12.21) for 



5i(f,dg) 



1 



infG 2 



(<W(G ; L i+1 (/))(^) + ll^+l(/)ll $G l ®G l {dg)) 



(2.9) 



We mention that in this case the parameter xi defined in (12. 3ft is such that 

X,<2supG?/infG? 
Assume we are working on path spaces 

S (l) ._ ( S (l-i) x s q where Gl = gr/ ( x /) 

and CSj)i>o are finite spaces. If we use for (x, dy) the independent Metropolis-Hastings 
kernel (|1 JOp . ([2^ is satisfied as < 1; e.g. HU theorem 2.1.]. Additionally, we have 



< 



/ (MP( Xl ,d yi ) - MjP(xi,d yi )jf(yi) 
So (I2.5P is satisfied for 



+ 



/0*-")(*«-i)( 1A fe^j) 



/ ^ / G' 

/ T^' L 1W 1A c £ ) 



(9) + 11/11 ^ 



1A 



(5) 



g ;-i(*;-i> 



where 



Ti,u(f,dg) 



+ 



< 2 



1 A 



G'1-1 



G'i-i{ x 'i-ij 



£ 

r' PS" 



1 A 



G1-1 



G'l-i( x 'l-i< 
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Interacting annealing models. To establish (|2,ip - (|2.2p . we can proceed similarly to 
Feynman-Kac models. It is sufficient to substitute Li + xK €i i + i to L; +1 in (|2.8p - (|2.9p . In 
this context, the Markov transitions given in (jl.lip are such that 

M®{x,dy) > (1 -e) Vi-iMLtidy) /5(Mf) < e . 

so (12. 4p is satisfied with ni = 1 and m;(l) < e. Moreover, we have 



(/) = (l-e) [^(/i) -*,_!(!/)] !,,(/) 



Using the decomposition (|2.7p one proves that 

1 - e 



(/) 



< 



infGn 



Hence condition (j2T5]l is satisfied with r i)A1 (/, dp) = infG; ' ; _ i 5 Sl _ 1 ^ Ll ( f )(dg) and Aj < (1 - e)/inf 



2.2 A multivariate central limit theorem 

To describe precisely the fluctuations of the empirical measures rfn around their limiting 
value 7r®, we need a few additional notations. We denote by D k i with < k < I the 
semigroup associated with the bounded integral operators D k introduced in (|2.ip . More 
formally, we have 

VI < k < I D Kl = D k D k+1 ... A 

For k > I, we use the convention D k i = Id, the identity operator. 

Using this notation, the multivariate CLT describing the fluctuations of the i-MCMC 
algorithm around the solution of the equation (jl.ip is stated as follows. We remind the 
reader that the integral operator Di + \ may depend on the measure tt^ . 

Theorem 2.1 For every k > 0, the sequence of random fields (£7n ) n >o on B(S^) defined 
below 



[/« : = V^TT 



converges in law, as n tends to infinity and in the sense of finite dimensional distributions, 
to a sequence of Gaussian random fields U( k > on B(S^ k ') given by the following formula 



:= E ^ VMB ik _ l)+1>k (2.10) 

0<l<k 

Here {V^\ >Q stands for a collection of independent and centered Gaussian fields with a 
variance function given by 

E(y(W) 

(2.11) 

= [(/-vr«(/)) 2 ] +2E„>i- W [(/"-«(/)) (M^U)" (/-.«(/))_ 

In the special case where M]p{x, .) = for all I > 1, that is E(U^(/) 2 ) = 

it [(/ — it (/)) 2 ] i the result corresponds to the one obtained previously in [6]. This 
special class of i-MCMC algorithms behaves as a sequence of independent random variables 
with distributions ^ l \rjn ) given by the local invariant measures of MCMC chains with 
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transition kernels M^i,,^,.). In the more general case considered here, the additional 

terms on the right hand side of (|2.1ip reflects the fluctuations of these MCMC algorithms 
around their limiting invariant probability measures. 

Finally we note that, if it was possible to sample exactly from tt^^ 1 ^, then we would 
have = V^ k \ However, we need to approximate 7r( fc_1 ) using an MCMC kernel which 
itself relies on an MCMC approximation of 7r( fe ~ 2 ) and so on. The price to pay for these 
additional approximations appears clearly in (|2.10jl . 

A Toy Example. Consider a Feynman-Kac model where := (S 1 ^ -1 ) x S) with S = 
{1,2} and 

G\ (1) = p(A+i-A), G\ (2) = gCA+i-ft) 
for p = 1 — q > 0, (A)i>o * s an increasing positive sequence and 



ji 



1 - 7T l+1 (2) TTl +1 (2) 
7T, +1 (1) 1-7T, +1 (1) 



where 



ir l+1 (1) = 1 - Tii+i (2) 



pPi+i _|_ qPi+i 

We have 7T/ +1 Lj +1 = 7T; +1 with 717+1 = (717+1 (1) 7T/ +1 (2)) and it is easy to check that the 
resulting Feynmac-Kac probability measure 7r^ +1 ) on S l+2 admits as a marginal distribution 
7i7 + i a t time l + l. For sake of illustration, we derive the expression of all the terms appearing 
in the variance of (/) in equation (l2~10l) of Theorem [2J] 

In this scenario, we have established in section 12.11 that -D/+i = D^L^i with D', = 
S[ n y) /V® (G[) = S l /717 (G[) where, recalling the notation X[ = (x' Q , . . . , x[), we obtain 

S l>w(l) (x llVl ) = G\ (x\) 5 Xl ( yi ) + (1 - GJ (sQ) ^ 

= s xi (yi) l'i+i (y'hy'i+i) 



so 



A+i = — (arQ 4, (w)^+i (l/U+i) + (1 - G\ (x'i)) tt^ 1 ) (y l+1 )\ . 

Finally the Markov transition kernel M^(xi,yi) satisfies 

Clearly even in this toy example, the expression of the variance of (/) is unfortunately 
analytically intractable. It is additionally only possible to compute numerically this variance 

(k) 

for very small values of k as the semigroup and the transition matrix would have 

to be computed for a number of values increasing exponentially fast with k. ■ 

Since the original version of this paper [5], the authors have become aware of a recent 
paper of Yves Atchade [4] which analyzes the local fluctuations of some related algorithms; 
namely the importance-resampling MCMC algorithm which is an interacting annealing 
model as described in section 11.31 and a version of the equi-energy sampler [12]. In [3], 

(k) 

the author provides a CLT associated to some random measures tt^ which converge almost 

(k) 

surely to tt^ as n tends to infinity. We provide here a multivariate CLT valid for any I (and 
not only 1 = 2). The variance expression we obtain depends explicitly on the first order 
semigroup D^.i and the fluctuations of the i-MCMC algorithm on lower indexed levels. 
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3 On the fluctuations of time inhomogeneous Markov chains 



To establish the proof of our main result (theorem 12. ip , it is necessary to first provide some 
general results about the fluctuations of time inhomogeneous Markov chains with Markov 
transitions that may depend on some predictable flow of distributions on some possibly 
different state space. 

3.1 Description of the model 

We consider a collection of Markov transitions M„ on some measurable space (S, S) indexed 
by the set of probability measures rj G V(S'), on some possibly different measurable space 
(S',S'). We further assume that there exists an integer uq>0 such that 

m(no) '■= sup /3(M"°) < 1 and we set p(no) := 2uq/{\ — m(no)) (3-1) 
■nev(s') 

We also assume that for any pair of measures (n,/i) G V{S') 2 we have 

\\[M^-M v ]{f)\\< f \[fi-v](9)\r^f,dg) (3.2) 

for some collection of bounded integral operator from B(S) into B(S'), indexed by the 
set of measures u G V(S') with 

sup / T^(f,dg) \\g\\ < A ||/|| for some finite constant A < oo 

fJ.GV(S') J 

We consider an increasing sequence of ex-fields (.F n ) n> g on some probability space (fi, J 7 , F). 
We let rj n be a (5")- valued random process adapted to the filtration T n (i.e. each prob- 
ability distribution rj n is J-" n -measurable) . We further assume that T n contains the a- field 
generated by the random states X p from the origin p = up to the current time hori- 
zon p = n of an S'-valued non homogeneous Markov chain X n with a prescribed initial 
distribution v G V{S), and some transitions defined by 

Vra > P(X n+ i G dx | F n ) = M Vn (X n , dx) (3.3) 

For example, T n could be a [(X' p ,X p ), < p < re), i.e. the canonical sigma field associated 
with the (5" x S')-valued process (X' n , X n ) n >o, and rj n — ^pjy $^p_g <5x£ is the flow of 
occupation measures of (X^) n>0 . In this context, (|3.3p reflects the fact that, given (r/ n )„>Q, 
the process (X n ) n>0 is a Markov chain with random Markov transitions defined in terms of 
the occupation measures (r] n ) n >o- 

We further assume that the variations of (?? n ) n are controlled by some sequence of random 
variables r(n) in the sense that 

Vre > \\j] n — Jfo_i|| < T~(ri), and we set r(re) := r(p) (3-4) 

0<p<n 

For re = we use the convention r/^i =0, the null measure on S'. 

3.2 Regularity properties of resolvent operators 

The main simplification of conditions (13. ID comes from the fact that M v has an unique 
invariant measure 

<f>(ri)M v = <!>(r)) G V(S) 
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In addition, the so-called resolvent operators 

P v : / € B(S) -> P„(/) := J] K - <%)] (/) e B(S) 



(3.5) 



n>0 



are well denned absolutely convergent series that satisfy the Poisson equation given by 

(M v -Id)P v = (*(»/)- Id) 







Resolvent operators are classical tools for the asymptotic analysis of time inhomogeneous 
Markov chains. In our context the Markov chain interacts with a flow a probability measures. 
To analyze the situation where this flow converges to some limiting measure, it is convenient 
to study the regularity properties of the resolvent operators P^ as well as the ones of the 
invariant measure mapping Q{rj) associated with M„. 



Proposition 3.1 Under the regularity conditions \3. 1\) and we have 

sup HP, || < p(n ) 

r?GP(S') 



(3.6) 



In addition, for any f £ B(S) and any (fi,n) £ V(S') we have the following Lipschitz type 
inequalities 

-*(/*)](/)!< / \[v-tAi9)\^M>dg) (3.7) 



and 



[P v -P,](f)\\ < / \[ v -n]{g)\rjf,dg) 



(3.8) 



where (T^,T^) is a pair of bounded integral operators from B(S) into B(S') indexed by the 
set of measures fj, £ V(S') such that 



and 



\g\\ YJf,dg) < p(n ) A 



\g\\ r'Jf,dg)<p(n )(l+p(n )) A 



Proof: 

The first result (|3.6p is proved in [8]. For completeness, it is sketched here. We use the fact 
that 



n>0 



to check that 



and 



|P,(/)||<E osc ( M , n (/)) 



n>0 



l^(/)|| < 



n>0 



osc(/) 



l^ll < 2 E P>1 EZo 1 mt 1)no+r ) 
- i-«m;'») 
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The result (|3.6p follows straightforwardly. The proof of (j3.7[) is based on the following 
decomposition 

Mv) - *(/*)] (/) = {Mv) - m m + $(77) [M„ - M M ]} (/ M ) 

with := (/ — <3?(/u)(/)). Under our regularity conditions on the integral operators M^, we 
find that 

| [$(77) - $(//)] (/)| < | [$(r?) - M M (/ M )| + || [M„ - M„] (/ M )|| 

<\mv)-^)]M tl (U)\ + j r M (^,d 5 ) 

This recursion readily implies ()3.7p with the integral operator given by 

T M (/,d<7) :=^r M (M;(/ /t ),d 5 ) 

n>0 

Finally we observe that 

/ T M (/,^)<^ j \\g\\ r M (A^(/ M ),d 5 )<A J] I WWII 

n>0 n>0 

Arguing as above, we conclude that 



/ ||<?|| T /U (/, d 5 ) < A £ osc(M;(/)) < p (no) A 

n>0 



This ends the proof of (|3.7h . The proof of (j3.8|) follows the same type of arguments. We 
observe that 

P r ,-P^ = P„(M V - MJPr, + [$(/i) - $(r?)] P„ 
To check this formula, we first use the fact that M^P^ = P^M^ to prove that 

P^M^ - Id) = (Mft - Id)P„ = (*(/*) - Id) 

This yields 

P M (M M - Id)P v = ($(//) - 7d)P„ 
Using the Poisson equation and the fact that P^(l) = we also have the decomposition 

P^M V - Id)P v = P^*{rj) - Id) = -P„ 

Combining these two formulae, we conclude that 

P M (M, - M M )P, = [P„ - P„] - - $(77)] P„ 

This ends the proof of the decomposition given above. It is now easily checked that 



||[^-^](/)||<||(M M -M 1J )P M (/)|| + J \[ V -^(9)\T,(P^f),dg) 

< J \\M-V](S)\ {T ll (P ll (f),dg) + T ll (P fl (f),dg)} 

The end of the proof follows the same type of arguments as before. This ends the proof of 
the proposition. ■ 
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3.3 Local fluctuations of weighted occupation measures 

This section is concerned with the fluctuation analysis of the occupation measures of the 
time inhomogeneous Markov chain introduced in (|3.3p . In section [H we shall use these 
results to analyze the fluctuations of i-MCMC algorithms. The fluctuation analysis of this 
type of models is related to the fluctuations of weighted occupation measures with respect 
to some weight array type functions. 

Definition 3.2 We let W be the set of non negative and non increasing weight array func- 
tions w = (w n (p))o< p < n fi<n, satisfying the following conditions 

3m > 1 such that ^Jio^(O) < oo 

n>0 

with 

Ve e [0, 1] w(e) := lim w 2 n {p) < oo 

0<p< [en] 

and some scaling function w such that lini( (EOj(:i )_ s .(o+,i-)( t;;7 ( e o)> w{ei)) = (0, 1). 

We observe that the traditional and constant fluctuation rates sequences w n {p) = l/\/n 
belong to W, with the identity function ro(e) = e. In our setup it is necessary to introduce 
more general sequences; see proof of theorem 12.11 in Section HI 

Definition 3.3 We associate to a given weight array function w G W the mapping 

W : neM(S) N ^W(ri) = (W n (ri)) n > eM(S) N 

defined for any flow of measures r\ = (r/ ra ) n >o G V(S), and any n > 0, by the weighted 
measures 

W n {r]) = w n{p) V P 

0<p<n 

The next proposition presents a pivotal decomposition formula of the weighted occupa- 
tion measures in terms of a martingale on fixed time horizon with a negligible remainder 
bias term. 

Proposition 3.4 We consider the flow of random measures £ := (Cn) n£ N ^ A^(S') N defined 
for any n > by the following formula 

Vn>0 Cn = [Sx n 

For n = 0, we use the convention $ = v so that (q = [5x — v\. For any weight array 
function w G W, the weighted measures W n (() satisfy the following decomposition 

W n (C)(f)= E W *(P) AM p+ i(/)+L n (/) (3.9) 

0<p<n. 

with the martingale increments 

AM p+1 (f) = M p+1 (f) - M p (f) := (P^f^Xp+t) - M^P^f)^)) (3.10) 
and a remainder signed measure L n such that 

\\Ln\\ <w n {<d) (l+p(n )T(n)) (2 + p(n )) A 
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Proof: 

We let Pn„-i be the integral operator solution of the Poisson equation associated with the 
Markov transition M r/n _ 1 with an invariant measure (?? n _i). By construction, we have 

Cn(f) = [f(X n ) - <D (fh-l) (/)] = P Vn ^(f)(X n ) - M^iP^mXn) 

For n = 0, we use the convention P v _ 1 = Id and M rj _ 1 = v. The proof of (|3,9p is based on 
the following decomposition 

Uf) = Mf) + B n (f) + C n (f) + AM n+1 (/) 
with the random processes A n (f), B n (f) and C n (f) defined below 

Mf) :=[P nB -P nn . l ](f)(X n+1 ) 
BnU) := [P Vn -Af)(Xn) ~ P Vn (f)(X n+1 )] 
C n (f) ■■= [M Vn ~ M^] P Vn _df)(Xn) 
Using the Lipschitz inequality (j3.8H presented in proposition 13. H we prove that 

K(/)|<|| K-i^-i] (/)ll 

< J \[71n-Vn-i](9)\r iln (f,dg)<T(n)p(n )(l+p(n )) A \\f\\ 

In addition, using the Lipschitz regularity condition (|3.2I) . we also obtain 
|C„(/)|<|| [M^-M^iP^ifM 

< II J \[rin-Vn-l}(9)\ ^vMn-i (/)> dg) 

< r(n) A \\P Vn _ 1 1| H/ll <r(n) A p(n ) \\f\\ 
By definition of the weighted measure W n (Q, we have 

W n (():= ^2 Mp)Cp(f) 

0<p<n 

= £ w n(p) AMp+i(/)+ ^ w ft (p)(i4p(/) + 5 p (/) + C7p(/)) 

0<p<n 0<p<n 



From previous calculations, we have 



£ Wn(p)(A»(/) + Cp(/)) 
0<p<n 



< w n (0) r(n) A p(n )(2 +p(n )) 



Finally, we use the following decomposition 

Wn(p)B p (f)= £ [Mp)Pv P -Af){X p )-w n {p+l)P r)v {f){X p+l )] 



0<p<n 



0<p<n 



+ [Wn(p + l)-W n (p)} P Vp (f)(X p+l ) 

0<p<n 



with the convention w n (n + 1) = 0. This implies that 



Y Mp)B P (f) 

0<p<n 



< 2 Wn (0) H/ll +p(no) H/ll Yl [wn(p)-w n (p + l)] 

0<p<n 

= (2 + p(no)) ll/H w n (0) 



(3.11) 
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The end of the proof is now a direct consequence of formula (|3,lip . 



Now, we are in position to state and to prove the main result of this section. 
Theorem 3.5 Assume there exist a measure rj and some m > 1 such that 

V/GBi(5') E(\Vn(f)-v(f)\ m )<£m(n) with ^e m (n)<oo 

n>0 

We let V n := W n (C) be the sequence of random fields on B(S) associated with a given weight 
array function w G W and defined in k3.9\) . We suppose that w G W is chosen so that 
w n(0)r(n) tends to as n —> oo. In this situation, V n converges in law as n —> oo to a 
Gaussian random field V on B(S) such that 

V(/,<?) G B(S) 2 E(V(f)V(g)) = $(r?) [C v (f,g)] 

with the local covariance function 

C v (f,g) := M v [(P v (f) - M v P v (f)) (P v (g) - M v P v (g))} 

Proof: 

Using proposition 13.41 it is clearly sufficient to prove that the random fields 

0<p<n 

converge in law to the Gaussian random field V as n — > oo. To use the Lindeberg CLT 
for triangular arrays of M^-valued random variables, we let / = (/ l )i<i<d G B(S) d be a 
collection of d- valued functions and we consider the M rf -valued random variables (£)(/) = 
(Wn,(C)(/ l ))i<i<d- We further denote by the cr-field generated by the random variables 
X q for any q < p. By construction, for any functions / and g G B(S) and for every < p < n 
we find that 

EK(p) AM p+1 (/) | J>) = 
EK(p) 2 AM p+1 (/)AM p+1 ( 5 ) | = w n (p) 2 C' p (f,g)(X p ) 

with the local covariance function 

C' p (f,g) := M Vp [{Pr^Af) ~ M^P^Af)) [Pr„-M ~ M Vp P Vp _M)] 
Using proposition 13.11 after some tedious but elementary calculations we find that 

\\C p (f,g)-C v (f,g)\\<c( V ) l^-^ir^s),^) 

+ / H-V] (h)\T 2 v ((f,g),dh) 

with a pair of bounded integral operator T* i = 1,2, from B(S) 2 into B(S') such that 

f \\h\\ T v ((f,g),dh)<c(r,) \\f\\\\g\\ 

In the above displayed formula, c(n) < oo stands for a finite constant whose value only 
depends on the measure n. Under our assumptions, the following almost sure convergence 
result readily follows 

hm \\C'Jf,g)-C v (f,g)\\ =0 (3.13) 
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On the other hand, using proposition 13.41 for any function h £ B(S) we have the decompo- 
sition 

£ W l(p)(h(X p )-$( Vp _ 1 )(h))= w 2 n (p)AM p+l (h)+lL' n (h) 

0<p<n 0<p<n 

with the martingale increments AWL+if/t) given in (13. 101) . and a remainder signed measure 
U n such that 

||L' n || <u£(0) (l+p(no)r(n)) (2 + p(V» A 
Using a Burkholder-Davis-Gundy type inequality for martingales, we find that for any m > 1 



E 



0<p<n 



< a(m) j ^ u> n (p) 4 J osc(/t) 

. 0<p<n 



for some constants a(m) whose values only depend on the parameter m. Thus, we find that 



E 



w 2 n (p) AM p+1 (h) 

0<p<n 



< a{m) w n (0) j ^ w n (p) 2 J osc(/i) 

. 0<p<n 



Under our assumptions on the weight functions w, if we take h = Crj(f,g) then by (|3,13p 
we obtain the following almost sure convergence result 



lim Vw„(p) 2 C v (f,g)(X p ) = lim Vi»„(p) 2 ^p-xXC^/, </)) 



p=0 



p=0 



We now combine the regularity property (|3.7|) with the generalized Minkowski inequality to 
prove that for any function h € B(S) and any m > 1 



£;=o^(p) 2 Mv P -i)(h) - Hv)(h)} 

<YMP) 2 / E(\[ Vp ^- V }(g)r)^T,(h,dg) 

n J 



p=0 



< (p(n )A\\h\\) [ w n (0) 2 + ^2w n (p) 2 e m (p-l) 

P =i 



This readily implies that 



E 



J2 n p= Mp) 2 Mv P -i)(h) - Hv)(h)} 



<w n (0) 2 (p(n )A\\h\\) l + £e m (p) 
If we choose h = C^f^g), this yields the following almost sure convergence result 



lim Y^w ri (pf *fap_i)(C„(/,$)) = *07)(C„(/,(7)) 



p=0 
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To summarize, we have proved the following series of almost sure convergence results 

n n 

lim JZw n (p) 2 C p (f,g)(X p ) = lim Tw n (p) 2 G n (f,g)(X p ) 

p=0 p=0 

n 

= lim V»„(p) 2 $(^i)(C„(/,g)) 

n— !-oo ' — ' 
p=0 

= $( V )(C v (f,g)) 

Therefore, we also have the almost sure convergence result 

n 

lim y^w n (p) 2 E(AM p+1 (/)AM p+1 ( ff ) | F p ) = <5>( V )(C v (f, g)) 

p=0 

Since we have x /o< p < n w n (p) = w n (0) — > 0, as n — > oo, the Lindeberg condition is satisfied 
and we conclude that the sequence of random fields W^iC) defined in (I3.12j) converges in 
law to the Gaussian random field V as n — > oo. This ends the proof of the theorem. ■ 

We end this section with an alternative and simpler representation of the covariance 
function of the random field V presented in theorem 13.51 We have 

C v (f,f)(x) = J M v (x,dy) [P v (f)(y) - M v (P v (f))(x)] 2 

Using the decomposition 

P v (f)(y) - M v (P v (f))(x) = [P n (f)(y) - P v (f)(x)} + [P v (f)(x) - M v (P v (f))(x)] 

= [P v (f)(y) - P„(/)(aO] + [f(x) - Hv)(f)] 

and the fact that 

J M v (x,dy) [P v (f)(y)-P v (f)(x)] = [M v (P ri (f))(x)-P v (f)(x)] 

= -[/(*) -*(*/)(/)] 

we prove the formula 

C v (fJ)(x) = J M v (x,dy) lPT,(f){v)-PM(x)] 2 -[f(x)-$(rj)(f)] 2 

On the other hand, recalling that $(77) = $(r/)M„ and using again the Poisson equation we 
also have 

J $( V )(dx)M v (x,dy) [P v (f)(y)-P 11 (f)(x)] 2 = 2^(rj)[P v (f) (/ - 

and 

2$(r?) [P v (f) 

= 2 $(77) [(/-f(r?)(/)) 2 ]+2E n > 1 *(v) [*W - <%)(/)) (/-<%)(/))] 
Hence we have proved the following proposition. 

Proposition 3.6 The limiting covariance function presented in theorem \3.5\ is alternatively 
defined for any function f £ B(S) by the following formula 

= $(77) [(/ - $(77)(/)) 2 ] + 2En>i [(/ " Hv)(f)) M»(f - <&(»/)(/))] 
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4 A fluctuation theorem for local interaction fields 



4.1 Introduction 



This section presents the fluctuation analysis of a class of weighted random fields associated 
to i-MCMC algorithms. Following the local fluctuation analysis for time inhomogeneous 
Markov chains presented in section [3751 we introduce the following weighted random fields. 



Definition 4.1 We consider the flow of random measures 



VI > Vn > 



(0 



* w re? 



(0 



(!) 



,(0 



. VFe associate 



For n = 0, we use f/ie convention <3?0 = *^ so fwrf 5, 

to a sequence of weight array functions (w^)i>q £ W N i/ie following flow of random fields 
(Wi l) {5^))i> on the sets of functions (B(S®))i> 

VI > Vn > := ^ u4°(p) 4° 

0<p<n 



We observe that the regularity conditions (|2.4p and (|2.5p ensure that the collection of 
Markov operators and their invariant measures $^(n) satisfy the regularity conditions 
([34]) and (pT2"]l introduced in section [34j Also observe that the i-MCMC chain A"( /+1 ) is 

a time inhomogeneous model of the form (13.31) with a collection of transitions M^t 1 ^ that 
depend on the flow of occupation measures rjn associated with the i-MCMC chain at level 



I. In this scenario, condition (|3.4p is satisfied with 



Vn > 



\n, 



71 + 1 



and we have 

r«(n):= £ r«(p) = 2 £ 

0<p<n 0<p<n 

Finally, we recall that for any m > 1 we have 



p + 1 



< 2(1 + log (n + 1)) 



V/ > V/ e E(|r/«(/) - 7T«(/)P)£ < b(m) c(l) -=L= (4.1) 

Vn + 1 

for a collection of finite constants b(m) whose values only depend on the parameter m (see 
for instance [8] ) . Using theorem 13. 5[ we can prove that the random fields 

V®:=W®(8®) (4.2) 

associated with a given weight array function 10 W € W converges in law to a Gaussian 
random field as n — )■ 00 such that 



V(/, 5 ) G ^(S«) 2 E(y« (/) y(0( 9)) = ^(0 \c®{f t g) 



(4.3) 



Here the covariance functions C^\f\ g) are defined in terms of the resolvent operator P 



associated to the Markov transition M^ (i _ 1) and the fixed point measure $0(7r^ ^) = tt^ 
with the following formula 

C®(J,g) 



M 



(0 

.(i-i) 



,(0 



(0 



,(0 



The main objective of this section is to prove the following theorem. 
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Theorem 4.2 We consider a collection of weight array functions (w^)i>o G W N . In this 

situation, the corresponding flow of weighted random fields (IQ )i>o defined in QJ^ I, con- 
verges in law, as n tends to infinity and in the sense of finite dimensional distributions, to 
a sequence of independent and centered Gaussian fields (V )i >0 with covariance functions 
defined in 



Using this result, the proof of the multivariate centra limit theorem 12.11 follows exactly 
the same arguments as the ones we used in the proof of theorem 2.1 in (6j section 6]. 
Proof of theorem 12.11 : 

We let S k := SS^ 1 be the fc-th iterate of the mapping S : i] G M(S^f i-> S(rj) = 
(Mv))n>0 G M(S®)" defined for any r, = (7? n ) n > G M(S®f by 



Vn > S n (r)) = —— V ri p 
n + 1 



0<p<n 



We observe that the time averaged semigroup E> k can be rewritten in terms of the following 
weighted summations 



0<p<n 



with the weight array functions s„ := (s^(p))o<p<n defined by 

VA;>1 V0<p<n = E T~ZT) and s « } (p) := 1 

p<q<n ^ ' 

We also know from proposition 6.1 in [6] that 

lim - E = ( 2k ) ] / k]2 

0<<j<n 

and, for any > 1, the weight array functions defined by 



Vn>0 V0<p<n W W (p) := sg* (p) / /E ^ ^ 



belong to the set W introduced in definition! 

Moreover, using proposition 5.2 in [6], we have the following multilevel expansion 

v (k) _ n (k) = £ sg+U^-O) p (fc _ 0+1)fc + 4*) (4.4) 

0<l<k 

where H« = (E { n k \> is a flow of signed random measures such that 

Vm > 1 sup E(|HW(/)| m )™ < 6(m) c(Jfe) (log (n + l))*/(" + 1) 

/eBi(S(*)) 

Here b(m) stands for some constant whose value only depends on the parameter m. This 
multilevel expansion implies that 



Vfa + i) 
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with the weighted distribution flow mappings W^ k ^ associated to the weight functions 
and a remainder signed measure E„ such that 

sup E(\Z { n k) (f)\)<c(k) (log(n+l)) fe / V / (^+l) 
/eSi(S«) 

The proof of theorem 12.11 is now a direct consequence of theorem 14.21 ■ 



4.2 A martingale limit theorem 

This section is mainly concerned with the proof of theorem 14.21 We following the same 
lines of arguments as the ones used in section 13.31 devoted to the fluctuations of weighted 
occupation measures associated with time inhomogeneous Markov chains. 

First, we introduce a few notation. For any k > and any fj, £ V(S^ k ~ 1 ^), let be 

(k) 

the resolvent operator associated to the Markov transition ' and its invariant measure 
G V(S^). We also set 

p {k) {n k ) :=2n k /(l-m k (n k )) 

with the pair of parameters (n k ,m k ) defined in (|2.4p . 

Using proposition l3.4l we find that the weighted measures wi k \5W) satisfy the following 
decomposition 

W« («(*))(/) = £ w«(p)AM!!f 1 (/)+l4*)(/) 

0<p<n 

for any / G B(S^) with the martingale increments 

\ %-\ ' Vp-i J 

(k) 

and the remainder signed measure L„ which are such that 

< {^(0) (l + 2p( fc )(n fc )(l + log(n + l))) (2 + p<*>(n fc )) a} — 

We consider a sequence of functions / = (f l )i<i<d, with d > 1, and /* = (f k )k>o G 
n fc >o^(^ (fe) ), and we let = (W^(.f ))i<i<d be the Revalued and ^-adapted 

process defined for any I > and any 1 < i < d by 

M n \f) ■■= E wp^Kfi) 

0<k<l 

From the previous discussion, we find that 
with the J 7 ^™) -martingale j\4^ n \f l ) given below 

M { ;\n ■= E A M n) (f ) with AM k n \n ■.= £ 4%) &K k Uf) ( 4 -s) 

0<k<l 0<p<n 

and the remainder bias type measure = J2o<k<i such that 

lim ||£ ; {n) || = 

Theorem 14.21 is now a direct consequence of the following proposition (see for instance 
the arguments used in section 4.2 in [6]). 
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Proposition 4.3 The sequence of martingales Ai^ n \f) defined in |^.5[ j converges in law 
as n — > oo to an M. d -valued Gaussian martingale Mi{f) = (■Mi{f l ))i<i<d such that for any 
I > and any pair of indexes 1 < i,j < d 

(M(f),M(f%= K (k) [c {k) (f\f) 

0<k<l 

with the local covariance functions vr^ [C^ k \f\ f)] defined in fFQ . 
Proof: 

The proof of the proposition is along the same lines as the proof of theorem 13.51 First, we 
consider the decomposition 

l{n+l)+n 

Mt\h= £ vf B) (/) 

i=0 

where for every < i < l{n + 1) +n, with i = k(n + l) +p for some < k < I, and < p < n 

Vf\f) :=^\v) AM« x (/ fc ) 

We further denote by Q± the a- field generated by the pair of random variables (Xp , -X^+i) 
for any pair of parameters such that fc(n + 1) + p < i. By construction, for any flow 

of functions / = (/;)z>o and g = (gi)i> G ]lz>o B(S^) and for evei T < i < l(n + I) + n, 
with i = k{n + 1) + p for some < k < I, and < p < n, we find that 

e(v}%) | g£\) = o 
E(v w (/)v w (5) | gw^^)^ q<*)(/, fl )(xW) 

with the local covariance function 



r(*) 



r(fc) p(*) 



Under our Lipschitz regularity conditions (|2.4p and (|2.5p . proposition 13.11 applies to the 
mappings and the resolvent operators As in the proof of theorem 13.51 after some 

tedious but elementary calculations, we obtain 

\\ci k \f,g)-C^{f,g)\\ 



< c(k) 



' (*-!) _ (fc-1) 
'lp-1 71 



+ 



(h) ^ { Sk)< k -v{{fk,9k),dh) 



where T 



(k),i 



1,2, is a pair of bounded integral operators from B(S^ k >) 2 into 



B(S( k -V) such that 



/ 



T 



(k),i 

n (h) i7r O-i) 



i)((fk,9k),dh) <c(k) ll/fcllllfffel 
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Combining the generalized Minkowski integral inequality with (|4.ip we prove the following 
almost sure convergence result 

lim \\CM(f,g)-CW(f,g)\\=0 

p— >oo y 

Arguing as in the proof of theorem 13.5} we obtain the following almost sure convergence 
result 

n n 

Km Y. w n^P) 2 CW(f,g)(xW) = lim J>i%) 2 C^(f,g)(X^) 

p=0 p=0 

n 



iimJ2^\p) 2 <*> (fe) U P k -i ] )(c (k) (f,g)) 

n— >oo * — i \ ^ J 

p=0 



\im{M^\f)M (n \g))i=c\ k \f,g):= V 7r^(C^(f,g)) 



This yields the almost sure convergence 

0<k<l 

Using the same arguments as the ones used in the proof of theorem 4.4 in [6], we conclude 
that the Revalued martingale (/) converges in law as n tends to infinity to a martingale 
Mi(f) with a predictable bracket given for any air of indexes 1 < j,j' < d by 

(M(f),M(f j '))i=cl k \f,f') 
This ends the proof of the proposition. ■ 



5 Path space i-MCMC models 

The aim of this final section is to show that the multivariate CLT discussed in section [2] can 
be generalized directly to analyze the fluctuations of the occupation measures of (Xn )o<k<l 
around the limiting tensor product measure <8>o<jfc<j7T > f° r an y time horizon I > 0. To do 
this, we check here that the regularity conditions discussed in Section 12.11 remain valid on 
path spaces. 

Let us fix a final time horizon I for (II. ID and consider the path space model given by 
Vn > X® := (4°),. . . ,X®) € S® := ]J 

0<k<l 

For every < k < I we denote by r/W £ V(S^) the image measure of a measure 77 G V(S^) 
on the A;-th coordinate level set S^ k ' of the product space 5^ := E[o<fe<2 ^ • Using this 
notation, it is easy to check that X$ is an S'W-valued self-interacting Markov chain with 
transitions defined by 

F(X® +1 e dx I (xV-%< p < n ,X®) = M^X^dx) (5.1) 

Vn 

with the occupation measures rjn ^ and the collection of transitions M^ [ |_ 11 defined by the 

Vn 

following formulae 

1 n 

n + l U> Xp vl 0<k<l Vn 
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where x := (x°, . . . ,x l ) G 5^, dx := dx° x • • • x dx l and we have used here the conven- 
tion M^K = M'°l It is straightforward to check that (IO) coincides with the i-MCMC 

Vn 

algorithm associated to the limiting evolution equation 

^ = $11(^-1]) with 7rM:=-7rW®...®7r« 

and the invariant measure mapping 

$K : ^ G i — y : = vrW ® $«(/j( )) ® . . . ® sW^C" 1 )) G 7>(S W ) 

To describe the main result of this section, we need to introduce some additional notation. 
For any < k\ < fc 2 , we set 

S lk lt k2] .- Yl SW and -K [klM] :=® kl <k<k 2 ^ k) ^V{S^' k ^) 

k\<k<k2 

For any < k < I, any pair (/ii,/j 2 ) G V(S^°' k ^) x p(5[ fe + 2 ' i + 1 ]) and any integral operator D 
from into 

5 (fc+i) 

, we denote by /ii ® D ® /j 2 the operator from into 

(/ii (g) D (g) /i 2 )((xi,X2,x 3 ),(iyi x dy 2 x dy 3 )) = Mi(rfyi) D(x 1 ,dy 2 ) ^2(^3) 

where (xi,x 2 ,x 3 ) G SW = (fiP,*-!] g, ® a nd (yi,y 2 ,y 3 ) G = (S^ ® ® ( S'[ fc+2 ^ +1 ] ) . 

Proposition 5.1 For any I > 0, f/ie mappings <&M i/ie collection of Markov transition 
kernels M [t _^ satisfy the set of regularity conditions \2. \2.J$ , and \2. 5\) as long as the 

corresponding conditions are met for the marginal mappings t&W and the transitions M (l-i) 

where 1 < k < I. In addition, the mappings <£>1' +1 ] satisfy the first order decomposition ^2.1\) 
with bounded integral operators -Dfz+i] from into given by 



D[i+i] = ^ m ® A+i + ^ 7r[ °' fel ^ ® ^ 



[k+2,1+1] 

0<fc<Z 



Before presenting the proof of this proposition, we emphasize that this latter directly 
implies that the multivariate CLT stated in section 12.21 is also valid for the path space i- 
MCMC algorithm discussed above. In other words, for every k > 0, the sequence of random 
fields (t/f ] )„> on B(S^) defined by 



n 



1 n 



v p=0 



converges in law, as n tends to infinity and in the sense of finite dimensional distributions, 
to a sequence of Gaussian random fields defined as by replacing the semigroups 
Di lt i 2 and the limiting measures ir^ by the corresponding objects on path spaces. 

Proof of proposition 15.11 : 
With some obvious notation, we have 



v'n>\ (M^j =®0<k<l{M^_ 1} 



Using the fact that 



0<fc<Z 
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for any sequence of probability measures /j,^ k \r]^ G V(S^), with < k < I, we prove that 

» (MU)") <- E " («§-.»)") s m E < 1 

0<fc<Z 0<fc<Z 

as soon as 



n > n [z] := (V <fe<i n fe ) x ^1 + 



log (Z + 1) 



Ao<fc<jlog(l/m fc (n fc )) 

We prove the pair of regularity conditions (|2.ip and (]2,5p by induction on the parameter I. 
We use the notation Dm, Em, Em mm, nm and Tm ^ the corresponding objects introduced 
in the statement of conditions (12 . 1 h , (j2.4j) , and (12.5j) . The results are clearly true for m = 
with 

:=7r(°) and M [ ° ] := AfW 

In this case, we readily find that 

D[ ] = D = 0, Sp] = 0, m [0 ] = m , n [0] = n and r [0]j(U = r 0)A , = 

Assume now that the result has been proved at some rank I. For any measure fi on 

we denote by (J- 1 ^ and pfi^ its image measures on and 5®. In this 

notation we have 

*P+i]( At ) = $W(^-i])®$('+D( M (0) 

and 

M^+ 1] ((u,*;),dx x dy)) = Afg.yCu.ds) x M^\v,dy) 

for any G SP +1 1 = (5^ x S®) and G = (S® x After some 

elementary computations, using the decomposition 

[$['+1]^) (,rM)] 

= ^(ttP- 1 !) [$( 1+1 >0i«) - ¥ l+1 \ir^)] 

+ [$P1( M P-1]) - ^Pl^P- 1 !)] (7T«) 

+ [$M( M P-i]) - $M(7rP~ 1] )] ® [$( 1+1 >0i«) - $( /+1 )(vrW)] 

we find that the first order condition f|2. 1 1) is satisfied with an integral operator -Cfy+i] from 
into S'P" 1 " 1 ] defined for any / G by 

D [l+1] (u,d(x,y)) = & l \n^)(dx) D {l+1) (u,dy) + D {l] (u,dx) & l+1 \TT^)(dy) 
= TT^(dx) D {l+1) (u,dy) + Dm(u,dx) ^ l+1 \dy) 

Condition (|2.5p is proved using the same type of arguments. This ends the proof of the 
proposition. ■ 
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